Designing Proficiency Tests to Accredit Previous 
Knowledge in American and British Literature 
in a Bilingual Education Program 

Diseno de examenes de suficiencia para acreditar el conocimiento previo 
en literatura americana y britanica en un programa de educacion bilingue 


Cesar Julio Leon Chica* ** 
Catalina D’Costa Martinez” 
Gisela Franco Jacome” 

Universidad El Bosque, Colombia 


This article aims at identifying the kind of American and British literature tests that can be designed 
to allow students who enter a bilingual education program at a private university in Colombia to have 
their previous knowledge in these two subjects accredited through a proficiency test. Students’ needs, 
opinions, beliefs, existing commercial tests, the University’s (specifically, the one where the study 
was conducted) literature programs, several anthologies, and competences required in the education 
program were all taken into consideration. Then the tests were developed, piloted, and validated with 
a focus group composed of ten students of said program. The results indicated that students require 
previous knowledge, literary competence, and command of the English language because those are 
determining factors in successfully passing the tests. 
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Introduction 

Literature has always been one of the strongest 
interests in the study of bilingual education. This 
leads to the focus of this research study which aims 
to find out what previous knowledge in American 
and British literature undergraduate students in 
the bilingual education teaching program bring 
with them to the university The question of which 
process would be appropriate for the accredita¬ 
tion of this previous knowledge arose. According 
to Webster’s Third New International Dictionary 
(1986, p. 13), “Accredit is, 1. to put (as by common 
consent) into a reputable or outstanding category: 
consider, recognize, or acclaim as rightfully pos¬ 
sessing uncontested status; 2. to give official au¬ 
thorization to or approval of”. On the other hand, 
accreditation is defined as “the act or process of 
accrediting” which means that the knowledge stu¬ 
dents already had when they entered a university 
program could be validated in either of two pos¬ 
sible ways described in the university’s student 
handbook: The first, homologation, is the equiva¬ 
lent of transfer credits and gives the same amount 
of credits for a subject studied at a different univer¬ 
sity or institution (Ministerio de Educacion Nacio- 
nal [MEN], Decree 0808 April 25, 2002, Articles 3, 
4 & 8). The second is a proficiency exam, which 
allows the student to prove his/her knowledge in 
a certain area by taking an exam. This has its legal 
foundations in Law 30 (Ley 30), Articles 20 & 28, 
authorizing each university to develop its own stu¬ 
dent handbook where it specifies how it is going 
to handle different academic aspects (Colombian 
Congress, 1992). 

The interest in this particular topic arose from 
our experience as English teachers and some 
of the Education students who also possessed 
a background of previous knowledge in both 
American and British literature. Since they could 


not get their previous knowledge accredited, it 
was proposed that the University let us design 
the two proficiency tests to be used as a tool to 
accredit previous knowledge in those subjects. 

The researchers’ goals were to open doors in 
the Faculty of Education of the University that 
would allow students’ previous knowledge to be 
taken into account and allow future generations 
to be able to sit exams on the subjects which they 
considered themselves proficient in. Accredita¬ 
tion saves students valuable time and, therefore, 
enables them to study new subject areas, thus the 
students of the Bilingual Education Program at 
the University will optimize their time and their 
knowledge by getting the credits in these subjects 
accepted. When adults are studying and working 
full time, besides being parents and having other 
family obligations- as is the case of most of the 
students at this university- time is precious. 

The Research Questions 

Our concern about this topic has evolved, and 
in order to take appropriate action, it was decided 
we should investigate this subject in depth. After 
having studied the problem, the following research 
question was posed: How can the University’s 
bilingual education students’ previous knowledge, 
the existing literature proficiency tests, and the 
University’s current American and British literature 
programs be taken into consideration in designing 
proficiency tests for these two subjects? Hence, we 
formulated the following specific questions to find a 
solution to said problem: What previous knowledge 
of American and British literature do students 
bring to the university? What type of tests can be 
designed to evaluate students’ previous competence 
in American and British literature? How can the 
criteria of the topics be tested and developed? 
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Literature Review 

Imagine that instead of reading literature, you 
heard it told to you by storytellers around a 
campfire or a fireplace. Telling and listening to 
make up stories, rather than writing them down, 
is part of the oral tradition. Eventually, versions of 
these tales were written down. 

(Carroll, Feldman, Kinsella, 
Stump & Wilson, 2002, p. 10) 

Folk tales travel wherever people travel. 

(Anaya, 2002, p. 10) 


After establishing the legal foundations to ac¬ 
credit the previous knowledge through the pro¬ 
ficiency tests, we focused on defining the four 
important constructs of our research: proficiency 
tests, literature, prior knowledge, and competence. 

Proficiency Tests 

According to Hughes (1989), “proficiency tests 
are designed to measure peoples ability in a subject 
regardless of any training they may have had in that 
subj ect. The content of a proficiency test, therefore, is 
not based on the content or objectives of any courses 
which people taking the test may have followed” (p. 
9). If this is so, then theoretically anybody who has 
ability to perform in a specific subject is entitled 
to take a test to prove their abilities. In this case, 
knowledge and previous knowledge are not taken 
into account. “Rather, proficiency is based on a 
specification of what candidates have to be able to 
do in order to be considered proficient” (Hughes, 
1989, p. 9). In Webster’s Third New International 
Dictionary (1986), the definition of proficient is the 


following: “to be well advanced in an art, occupation, 
skill or a branch of knowledge: unusually efficient”. 

One of the proficiency tests for literature 
recog-nized worldwide is the Graduate Record 
Exa-mination (GRE) subject test (literature in 
English), which is not for undergraduate students, 
of course. Even though it is a useful tool for com¬ 
parative purposes, the researchers decided that 
the Subject Area Test (SAT) literature tests recom¬ 
mended by College Level Examination Program 
(CLEP) are more adequate to the level and needs of 
the participants and the University. 

Regarding the abovementioned data, Hilliard 
(1999) asserts that “All standardized tests have their 
place, even the weak ones. Without them we would 
have nothing —no way to hold people accountable 
for teaching or learning” (p. 38). In connection with 
this, our goal was to design the proficiency tests 
in order to account for students’ competences in 
literature through the mentioned tests. Regarding 
tests, Hilliard (1999, p. 63) asks: “Then why use 
standardized tests at all?” Schools and districts use 
them because they provide data and results that are 
essential for improvement. For these reasons, we 
took the theory and some models as test references. 

Student performance data are essential elements 
in a healthy and successful improvement effort 
(Glickman, 1993; Fullan, 1991). These authors state 
that standardized tests, despite some shortcomings, 
provide numerical and intelligible data as to how 
well a learner is performing or improving, and 
vital information about patterns of strengths and 
weaknesses among students. 

Literature 

To define this construct (a subject that focuses 
on special topics because it is content-based), we 
can start with Long (2005), who says that literature 
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is the written record of man’s best thoughts and 
feelings, and English literature is the part of that 
record which belongs to the English people. In its 
broadest sense, literature includes all writing, but 
as we commonly define the term, it excludes works 
which aim at instruction and includes only the 
works which aim to give pleasure, and which are 
artistic in that they reflect nature or human life in 
a way to arouse our sense of beauty (Long, 2005). 

According to the Encyclopedia Britannica 
(1994, p. 398), literature is a body of written works. 
The name is usually applied to those imaginative 
works of poetry and prose distinguished by the 
intentions of their authors and the excellence of 
their execution. Literature may be classified into a 
variety of categories, including language, national 
origin, historical period, genre, and subject matter. 
When we study literature we deal chiefly with 
the great, enduring books, which may have been 
written in an older or a recent time, but which have 
in them timeless magic. In order to be considered 
proficient in literature, a person must understand 
each form of writing and be able to identify each 
text in its corresponding genre. 

Prior Knowledge 

As already said, this investigation intends 
to determine the role prior knowledge plays in 
determining what type of test will be designed in 
order for students to be acknowledged proficient 
in American and British literature. 

According to Gonzalez (2000), “what we don’t 
know prevents us from learning”. The author says 
that many research studies uphold the importance 
of a reader’s prior or previous knowledge. It is a 
fundamental variable in reading comprehension 
and in learning from texts. 

The researchers agree with him, and think that 
this is especially applicable when referring to lit¬ 


erature. Besides, Gonzalez (2000) states that there 
are special tasks focused on activating previous 
knowledge. In connection with this concept, Hito- 
tuzi (2005) mentions that “Students are not a tab¬ 
ula rasa on which a supercilious teacher can record 
their knowledge; instead, they have knowledge and 
experiences of life and language (as well as other 
subjects, contents, and topics) which can contribute 
greatly to the learning process” (p. 98). In our case, 
some students of the Bilingual Education Program 
had some previous knowledge and experience in 
literature and thus wanted this knowledge to be ac¬ 
credited through a proficiency test. 

Hughes (1989) states that prior knowledge and 
content seem to have a certain degree of influence 
regarding proficiency; thus, he supports the idea 
that if students have previous knowledge of a topic 
-in our case, literature- it will affect their test 
results. Similarly and following Roschelle (2004), 
it can be stated that it would be very difficult to 
learn without prior knowledge because students 
construct new concepts from prior knowledge. 
Since prior knowledge is based on experiences, it 
can be said that new knowledge is also defined by 
the new experiences in life. 

Rico (2005) adds that “The level of proficiency 
is seen according to the students’ capacity to 
recognize and master the elements of the system” 
(p. 99). Furthermore, he links several important 
concepts related with test design: proficiency, test 
design, competence, and previous knowledge. These 
elements supported our main goals in designing 
the proficiency tests in which, first, we found 
out what previous knowledge of American and 
British literature the participants had. In doing 
so, we followed Hughes (1989), who emphatically 
points out that the first step in test designing or 
construction is the statement of the problem. The 
test must be “perfectly clear about what it is one 
wants to know and for what purpose” (p. 48). 
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Competences 

According to the Colombian National Ministry 
of Education, competences are a set of knowledge, 
attitudes, disposition and skills (cognitive, social- 
affective and communicative), all interrelated to 
help students in the learning process, including 
the development of a new sense of activity in new 
and challenging contexts (MEN, 2004). Therefore, 
competence implies knowing, being, and knowing 
how to do. From the above, competences can 
be seen as the battery of knowledge, skills, and 
attitudes needed to carry out a determined activity 
and/or to solve problems in an autonomous and 
creative manner. 

According to Smith & Wilson (1990), “Within 
Chomskyan theory, there are two levels of assess¬ 
ments: the first is called the level of competence, 
the speaker’s knowledge of language which is purely 
linguistic, and the second is called the level of per¬ 
formance, the speakers use of language” (p. 20). 

Competences play an important role in the 
program under study. When a student wants to 
accredit previous knowledge in literature, s/he 
should not only demonstrate competence in the 
English language, but in the literature subject. He 
needs to perform and accomplish tasks in regards 
to aspects such as the ones mentioned before: 
national origin, historical period, genres, and 
outstanding works, among other aspects of the 
subject matter. 

Research Design 

Type of Study 

The study was a descriptive qualitative re¬ 
search because “the researchers do not set out 
to test hypotheses, but rather to observe what 


is present with their focus, and consequently 
the data, free to vary during the course of the 
observation” (Larsen-Freeman & Long, 1993, p. 
10). According to these authors, this study follows 
a cross-sectional approach where the performance 
of a large group of subjects is studied, and the data 
are usually collected in one session. Hence, we 
gathered the data through personal answers of the 
interviewees by means of questionnaires and oral 
interviews. 

According to the above-mentioned authors, 
“one of the characteristics of the qualitative 
paradigm is that it is not generalizable; it is 
process-oriented, subjective, grounded, discovery- 
oriented, exploratory, descriptive, and inductive. 
It is also holistic and assumes a dynamic reality” 
(Larsen-Freeman & Long, 1993, p. 11). First of all, we 
wanted to find out if the participants had previous 
knowledge, and if so, if they were willing to take 
the tests to accredit their previous knowledge, to 
then be able to design the literature tests. 

The Participants and Setting 

The study took place at a private university 
in Bogota. The undergraduate program is called 
B. Ed. Program in Bilingual Education with Em¬ 
phasis on Teaching English (Licenciatura en 
educacion bilingiie con enfasis en la ensenanza 
de ingles). It is one of the five options offered by 
the Faculty of Education. At the time that it first 
opened, it was the only Faculty of Education in 
Colombia which offered a program specifically 
in bilingual education. This program has an af¬ 
ternoon schedule which leads us to think all its 
students are not recently graduated from high 
school. Part of the population registered for this 
program is comprised of teachers who have many 
years of experience and want to increase their 
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professional standing. Others are young teachers 
who need to work while studying for their degree. 

Out of sixty four students of the Faculty 
of Education, ten students were selected as a 
focus population to answer a prior information 
questionnaire (See Appendix 1). They replied that 
they did have previous knowledge in American 
and/or British literature, and also answered 
affirmatively to the question of wanting to present a 
proficiency test to have this knowledge accredited. 
Their ages ranged from sixteen to fifty five. Some 
of them studied at bilingual schools in the city; 
others either studied or lived for a time (varying 
with each individual) in English speaking countries. 
Other students gave account of their previous 
knowledge in these areas due to personal interest 
and autonomous study. 

Instruments and Procedure 

As already said, at first we developed a Prior 
Information Questionnaire (See Appendix 1). 
We selected a questionnaire to collect the first 
information because it let us gather data in one or 
two sections. As Burns asserts, questionnaires are 
“written sets of questions used to gain responses in 
non face-to-face situations; questions are usually 
focused on specific issues and may invite either 
factual or attitudinal responses” (1995, p. 117). 

After selecting a focus population, an interview 
was designed by the research team to ask the target 
population their opinions, beliefs and experiences 
with previous proficiency tests (See Appendix 2). 
The information was obtainedby directly asking the 
chosen and consenting participants, and recording 
their answers with an audio tape recorder; it was 
then transcribed in the computer for analysis. We 
bore in mind Siedman (1991), who, concerning 
interviewing as qualitative research, states that 
“the purpose of in-depth interviewing is not to get 


answers to questions, nor to test hypotheses, and 
not to ‘evaluate’ as the term is normally used; at 
the root of in-depth interviewing is an interest in 
understanding the experience of other people and 
the meaning they make of that experience” (p. 19). 

Those two instruments were chosen after 
reflecting on the type of question to be answered 
and the type of research that was being carried 
out. The information was typed into the computer, 
then organized; lastly the researchers carried out 
the analysis by closely following the steps that dealt 
with levels of analysis. 

In the Literature Review section it is stated that 
one of the main limitations of proficiency exams or 
standardized tests is the fact that they are designed 
to be taken by large groups of people at one time 
and, therefore, must necessarily be machine scored 
(Barr, Craig, Fisette & Syverson, 1999; Hilliard, 
1999; Covey, 1989; ETS, 2002). Thus, commercial 
standardized tests were not a good option for our 
purpose. To avoid this limitation, the researchers 
took it upon themselves to design two “tailor-made” 
literature proficiency tests. In this task we considered 
the students’ needs, opinions, and beliefs, as well as 
the Faculty’s requirements for the above-mentioned 
subjects. Then the tests were developed and piloted 
with the previously chosen candidates. 

In the designing of the tests we took into 
consideration the data provided by the participants 
through instruments described above, as well as 
information from several literature textbooks, 
anthologies, the GRE, the CLEP, and the SAT. All 
these documents are related with literature subject 
tests and were used as references to design the two 
literature tests. 

The participants took the tests. Then, both 
tests were assessed (graded) by the researchers 
and a final questionnaire was presented to the 
participants in order to learn their opinions about 
the literature tests (See Appendix 3). 
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Data Analysis 

After the ten interviews were carried out, they 
were transcribed uniformly in order to be analyzed 
with the computer software Atlasti. This program is 
a tool designed with the objective of facilitating the 
qualitative analysis of large quantities or volumes of 
data, or information which has been gathered for 
research purposes. It is not meant to replace the 
human interpretation of the analyzing process, but 
its main purpose is to help to accelerate many of 
the activities implied in the qualitative analysis and 
interpretation such as text segmenting, quoting, 
and coding. 

Students' Opinions Regarding 

the Type of Test That 

Should Be Designed 

The use of this program allowed the researchers 
to define certain categories based on the common 
patterns that resulted from the analysis of data. 
These patterns provided both quantitative and 
qualitative information about the students’ opinions 
regarding the type of test that should be designed 
to evaluate or to accredit their previous knowledge 
in literature. The categories found were previous 
knowledge, reasons for taking the test, type of test, 
number of questions, and testing time, as can be 
seen in Table 1. These categories were defined by 
taking into consideration the number of times the 
target words were mentioned by each interviewee 
when we used the format shown in Appendix 2. 

The categories shown in Table 1 had, in turn, 
several subcategories. With the purpose of designing 
the literature proficiency tests, the researchers had 
to find out what the students thought the focus 
of proficiency test should be. There were three 
questions in the interview that focused on these 
aspects: One theory of the two literature tests 


previously mentioned, CLEP and SAT, considers 
that a well prepared examination may include 
the following: authors, texts, historical periods, 
genres and figurative language, among others. 
The students’ answers reflect the same, since the 
interviewees said that they would like tests about 
authors, topics, historical contexts, different genres, 
and a combination of the above. 

‘Reasons’ was an important category for us, 
due to the need to know why the students wanted 
or were willing to take a proficiency test. The 
main motivations observed were to save time and 
to advance subjects in their studies. It should be 
noted that when the surveys and interviews were 
analyzed, the answers confirmed that the existing 
reasons stated above for taking proficiency 
exams were widely shared by the students at the 
University. The Faculty’s Dean and Academic 
Council understood these reasons and let us carry 
on with the design and validation of the tests. 

‘The students’ interviews regarding type of 
questions revealed that they did not want closed 
questions based on memory. They expressed that 
they wanted short, open-ended questions, essays, 
and reading comprehension. 

The majority of the students interviewed an¬ 
swered that the number of questions should be 
between 1 and 20. As to the number of questions the 
test should have, 8 students said that they wanted 
from 1 to 20 questions; one student said that it 
should have from 1 to 10 questions; three students 
said from 1 to 50, and one student said from 1 to 
100. However, taking into account the extent of the 
topics to be examined, and comparing it with the 
number of questions of the existing standardized 
exams, we considered that this was a very low 
number, as the average number of questions in 
existing literature proficiency examinations for 
undergraduate students is about one hundred. The 
GRE has 230 questions, but this is only a reference 
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Table 1. Students' opinions regarding the type of test that should 
be designed to evaluate their previous knowledge in literature 


Categories 

Sub-categories 

Interviewees' Responses* 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

TOTAL 

Previous 

Knowledge 

Analyzed texts 






3 

1 




4 

Authors 

4 

2 


3 

4 


3 

1 

4 

2 

23 

Biographies 


1 









1 

Genre 

4 

3 



1 

1 




1 

10 

Historical contexts 

3 




4 


2 




9 

Topics 

2 



1 

3 


2 

1 

1 

1 

11 

Reasons 

Advanced subjects 





1 


1 




2 

Motivation 




1 

3 


1 




5 

Negative motivation 

2 










2 

No previous knowledge 



1 





1 



2 

Save time 

2 




2 


1 




5 

Text analysis 

1 










1 

University’s duty 

1 




1 






2 

Test type 

Closed questions 









1 

1 

2 

Combination 

1 

1 

1 

1 

3 



1 


1 

9 

Essay 

1 

2 

1 







1 

5 

No closed questions 

1 










1 

No memory 

1 

1 









2 

Reading comprehension 

1 

1 

1 

1 

1 

1 

1 



1 

8 

Short, open-ended 
questions 

1 

1 



1 



1 



4 

Number of 
Questions 

1-10 


1 

1 








2 

1-20 

2 

1 

1 


1 


1 


1 

1 

8 

1-50 

1 



1 


1 





3 

1-100 








1 



1 

Testing time 

Any semester 

2 

1 

2 

2 

2 

1 

1 

1 


1 

13 

First semesters 

1 

1 




1 

2 

1 



6 


* The ten participants of the study were named with codes (from 1 to 10) to maintain their anonymity. 
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source as it is an exam for students who have 
completed a degree in literature and are seeking 
to obtain a master’s degree. This did not comply 
with our case, which was proficiency exams for 
undergraduate (education, not literature) stu¬ 
dents. Therefore, we had to focus on the SAT- 
CLEP examinations for accrediting literature as 
undergraduates, which is the nearest equivalent 
to what we were expecting to achieve in our 
program. Considering the scope of the University’s 
currently existing programs in American and 
British literature, equivalent tests, and students’ 
answers, we considered that the adequate number 
of questions should be around 60 or 70 because 
not all of them were going to be multiple-choice 
and we would include some short answer, open- 
ended questions, and one essay question. 

The category testing time included two sub¬ 
categories, and was aimed at finding out students’ 
preferences as to when, during their studies, they 
would want to take the proficiency tests. Of the 
total of answers given, the majority indicated 
that they would take the test any semester; and 
the others indicated that they would like the op¬ 
portunity of taking it during the first semesters (1- 
3) because what they had learned at school was still 
fresh in their minds. This question was solely for 
informational purposes because it had no usability. 
As noted by Brown, it “has to do with the degree 
to which a procedure is practical to use”. However, 
the university students’ handbook states that profi¬ 
ciency examinations may be taken at any time 
during the undergraduate studies (1995, p. 52). 

Taking into account that the objective of 
the study was to find out what the role of the 
previous knowledge was for designing a literature 
proficiency test, this was one of the most important 
categories. Questions one, two, and eight from 
the interview provided the greatest amount of 
information regarding this category (See Appendix 


2). Table 2 illustrates this, with answers given by 
three participants, about previous knowledge. 

The students’ comments in Table 2 supported 
the quote from the CLEP (2007) in which it states 
that only if they had this “broad knowledge of 
literature” and “a familiarity with the basic literature 
terminology”, the participants were able to obtain 
good results as was observed when the tests were 
applied. 

The Designing of the Proficiency 
Test and Its Outcomes 

Having analyzed all of the above information 
from our three sources, students’ interviews, theory 
(University’s current programs, standardized tests, 
and anthologies), and the researchers’ points ofview, 
we decided that the tests in American and British 
Literature should have 70 questions each and one 
essay. These questions were thirty multiple-choice 
(30 points), a time line (10), reading comprehension 
(20), ten open-ended questions (10), and one essay 
(30). The numbers in parentheses are equivalent to 
the number of points each type of question would 
count for a total of one hundred points (100). The 
minimum passing grade was 70/100. 

The authors were chosen from those included 
in the current University’s programs for each of 
the subjects, and the most well-known authors not 
included in these programs, but who were taken 
from high school literature anthologies. 

After having corrected the literature profi¬ 
ciency tests presented by ten students at the Uni¬ 
versity, the general results were as follows: half 
of the total number of students who took the lit¬ 
erature proficiency tests passed them- a majority 
passing British Literature and a minority passing 
American Literature. 

In British Literature a great number passed the 
time line, multiple-choice, and reading compre- 
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Table 2. Some responses about previous knowledge on literature 


Interviewees 

1. Would you like to take 
the proficiency test for 
American literature? Why? 

2. Would you like to take 
the proficiency test for British 
literature? Why? 

3. What do you remember about 
your previous literature studies? 

1 

Yes I would. Basically be¬ 
cause I do have some back¬ 
ground knowledge about 
this subject, and I think that 
coming here to that class is 
going to be... I know its 
going to sound rude, but its 
going to be a little bit of a 
waste of time in the sense 
that I’m not really going to 
learn anything new, and 
there’s going to be an incre¬ 
dible amount of time that 
I’m going to spend in a class 
doing things that for most 
of the part I pretty much al¬ 
ready know. So, I’m not rea¬ 
lly looking forward to ta¬ 
king the class. 

Well, I would have liked to 
take it, actually I’m taking the 
subject right now and it’s 
been pretty much the same 
thing I just said but regarding 
British literature, in the sense 
that I’m coming to class and 
listening to stuff that I al¬ 
ready know for the most part 
and making... I don’t know, 
writing essays on subjects I 
already know about, which I 
don’t really think I’m really 
using that time appropriately 
when I could be doing more 
useful things. 

I think more than the author, 
subject is concerned talking 
about particular periods of time. 
I’m really fond of drama, and I’m 
really fond in particular about 
renaissance drama, so everything 
about Shakespearean drama I 
like, and I’m also very into mod¬ 
ern fiction, things like... I really 
like that; we did take some 
classes on that back in school. 
Poetry. Particular authors, I’m 
not sure what I would say, things 
like Taylor, Swiff or John Milton, 
Shakespeare come to mind, 
Oscar Wilde... 

5 

Because I have a back¬ 
ground and knowledge in 
American Literature, and I 
believe that if I already know 
some of the topics and sub¬ 
jects a test will help me, to 
improve a given my, my, my 
grade on that particular 
subject. 

For the same reasons I have, I 
teach literature and I am 
aware of the different histor¬ 
ical periods, and authors, and 
most important novels, and a 
test will help me a complete 
my career better. 

Well, I remember a lot of things. 
Basically, I understand that lite¬ 
rature in different parts of the 
world has different contexts. In 
the United States we have diffe¬ 
rent authors, and in England we 
have different authors as well, and 
we also have to take into account 
Colombian literature; there are a 
lot of authors and periods that 
people study and try to under¬ 
stand in order to get the main in¬ 
fluence of authors’ writings in 
their time, to see if they have 
achieved a knowledge and un¬ 
derstanding of their culture and 
the times in which they are writ¬ 
ten, even presents writers or past 
writers. 

10 

Yes. Why? Well, because I 
have read various American 
writers, and I consider my¬ 
self knowledgeable on them. 

Yes. Why? For the same rea¬ 
son stated above. 

Now I am taking British Litera¬ 
ture, and I recall since Old English 
“Beowulf”, Elizabethan Period 
(Shakespeare), Samuel Taylor C. 
Wordsworth. 
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hension; less than half passed short opened-ques¬ 
tions; and everybody did well on the essay. Finally, 
all of the students achieved a passing grade on 
the test, (60/100), but since the minimum grade 
for obtaining the proficiency credits was placed at 
70%, most of the participants achieved it, but one 
person did not. The time allotted was adequate be¬ 
cause all of the students used fewer than the three 
hours given, and only one person took two hours 
and fifty minutes. 

In American Literature nobody passed the 
time line; one person passed the multiple choice 
section; one person passed the short open-ended 
question; and several passed the essay. One person 
did not complete it, and one person did not do 
it because she felt unable to do it. One person 
obtained a perfect score (30/30) on the essay. 

The time allotted was enough because all of 
the people who took the test needed less time to 
complete it. After the students sat the literature 
proficiency tests, they were asked to answer a final 
questionnaire (See Appendix 3) in order for the 
researchers to obtain feedback on the tests. Some 
aspects worth mentioning about the feedback 
results are the following: Six students did not study 
at all for the test. However, five of them passed 
the test, which supports the idea that previous 
knowledge is important when taking a proficiency 
test. Participant 5 said: “No studying; I just relied 
on previous knowledge”. 

Before the students knew their results, more 
than half thought they had done acceptably and 
the rest thought they had done poorly. The main 
reasons mentioned were lack of study and prepa¬ 
ration. Nevertheless, the tests results obtained 
were usually better than what the participants 
thought. For example, participant No. 2, who an¬ 
swered acceptably, did very well (93/100); likewise, 
participant No. 9, who answered poorly, obtained 
the passing grade of 70/100. 


Regarding the allotted time, three participants 
expressed that it was sufficient; four said it was 
adequate, and two thought it was excessive. 
In general, the students’ acceptance of the test 
was good even though they suggested some 
modifications. This can also be observed in the 
explanations given to question No. 7. Some of these 
explanations contradict each other. As none of the 
reasons given was supported by a considerable 
majority, the researchers concluded that the tests 
should not be modified. 

Findings, Implications, 
and Applications 

The tests were based on the type of questions 
that students wanted to have included as well as 
what the researchers determined from studying 
the university’s programs, the existing commercial 
tests, and several literature anthologies. This 
made it possible to combine both proficiency and 
achievement (theory-practice; previous knowledge- 
ability/competence) in a more communicative type 
test. In general the researchers thought that the 
most difficult part of the test was going to be the 
essay, but the students’ results show that all of the 
participants did well, even those who expressed 
that it was difficult and long. 

As researchers, we thought that the participants 
were going to do very well on the reading compre¬ 
hension part, as the answers can all be found in 
the text given if the participant has good reading 
skills, and because in the interview eight out of ten 
wanted it to be included in the test. However, this 
section did not produce the expected results. This 
may be due to the fact that many students read too 
fast, and did not pay sufficient attention to details 
during testing. 

The above has a definite relationship with what 
Glickman (1993) expresses in that proficiency tests 
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are useful in determining patterns of students’ 
strengths and weaknesses in subject areas. 

With respect to the short open-ended questions, 
the results obtained by many of the participants were 
poor. This was unexpected because the questions 
included in the tests were designed around very 
well-known topics. It was also strange because in 
the interview students answered (see Appendix 2) 
about the type of questions to be included in the 
test, three of the participants specifically asked to 
include this type of questions. 

It is important to acknowledge the fact that the 
profile needed for presenting these tests was that of 
having prior knowledge in either American or Brit¬ 
ish literature, or in both. Seven of the participants 
met this requirement either because they studied 
at bilingual schools or because they were teaching 
literature. This issue brings out the fact that there 
were three students who presented the proficiency 
tests without having the required profile. 

Taking into account the students’ feedback on 
the tests, the following aspects should be noted: 
“It should be based on the University program” 
(Participant 1); “Some authors are not in the 
program” (Participant 10). In connection to this, 
it should be kept in mind that the focus of a 
proficiency test is to validate the student’s previous 
knowledge in a given subject in order to accredit 
it; therefore, it must include a general overview of 
the subject area. 

A proficiency test is not intended to be limited 
to any course, curriculum, or single skill in the 
language. The following testimonies evidence some 
participants’ positive views about the tests: “It’s OK, 
better than the program” (Participant 2); “It was 
good, adequate to the topic” (Participant 4). 

Although the test was the result of analyzing 
carefully the number of questions, the time allotted 
and the topics of several literature commercial 


tests, other general comments were that it was “too 
long” (Participants 7, 8, 9, and 10). 

All the participants who passed with 70% or 
more had previous knowledge in literature from 
other studies either at high school, university or 
both, as was confirmed by the information ob¬ 
tained from the initial interviews. This finding was 
in addition to having taken the British literature 
course at the University in the semester just prior 
to taking the test. Another reason that influenced 
the above-mentioned difference was that all the 
students who took this exam and passed it had 
a high level of English and could be considered 
bilingual since they were taking or had already 
passed English VI at the University. This was a 
very important factor because the test requires a 
full command of the English language. 

On the other hand, none of the participants 
had taken an American literature course at the 
University. Additionally, the three participants with 
the lowest scores on the test were still doing the first 
levels of English at the University at the time they 
took the test. It was inferred, therefore, that they 
are not totally proficient in the English language 
which, as stated above, is a determining factor in 
the final result. This is illustrated with this excerpt: 
“I did not even try to do the essay because I did not 
understand what it was asking” (Participant 4). 

The experience learnt from this research 
project shows that previous knowledge must be 
validated in order to both let students save time in 
completing their career and having the possibility 
of studying other subjects. This can be supported 
with Gonzalez’s words (2000) when he expresses 
that many research studies uphold the importance 
of a reader’s prior or previous knowledge. It helps, 
empowers, and is a fundamental variable in 
reading comprehension and learning from texts. 

There is a definite relation between students’ 
profile (background) and previous knowledge in 
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order to obtain the accreditation of the latter, which 
is important to have in mind when the students 
ask to have their previous knowledge validated by 
presenting proficiency tests. If they do not have a 
good mastery of English, they will probably not be 
able to answer the essay. 

As the majority of the students who took 
British Literature passed the test, the researchers 
think that the test may be a useful tool for the 
University in the future. Although the results were 
different for the America Literature test, it can be 
explained through the observations stated above 
that the reasons for this could be attributed to the 
students’ lack of previous knowledge, and their 
low command of English, and not to failures in the 
test design itself. 

This research demonstrated that the three 
aspects mentioned in the initial research question, 
students’ previous knowledge, the existing literature 
proficiency tests, and the University’s current 
American and British literature programs, were 
carefully studied and included in the design of the 
two proficiency tests. Our general objective was 
also achieved despite what Cohen (1994) says about 
standardized tests. This author believes there is a 
difference between proficiency tests, which are used 
mainly for academic purposes because they focus 
on theory, and achievement tests, used mainly for 
administrative purposes because they focus on 
abilities or competences. 

Conclusions, Limitations, and 
Questions for Further Study 

The main conclusion after having carried out this 
research project was that both previous knowledge 
and competences are determining factors in the test 
results. The difference in the test results among the 
participants who had previous knowledge both in 
literature as well as mastery of the second language 


supports what the theories and studies mentioned 
before state, regarding testing. 

Another important finding was that compe¬ 
tences, evaluated through reading comprehension 
and essay writing, can also be taken into consid¬ 
eration when assessing both previous knowledge 
and competence in literature. 

Two literature proficiency tests were designed 
and validated although, according to TESOL litera¬ 
ture, teachers do little collaborative research and 
most of the literature regarding proficiency tests is 
limited to language proficiency testing. The tests 
designed in this study achieved a synthesis of as¬ 
sessment methods, as well as a more communica¬ 
tive approach to literature testing. As proposed by 
Hilliard (1999), they evaluate previous knowledge 
(theoretical subject content/literature) in the time¬ 
line, multiple choice section, and short open ended 
questions, as well as competences considered in the 
reading comprehension and essay writing. 

Some limitations we had to deal with in this 
research project were that ten people comprised a 
small population compared to the total number of 
bilingual education students (111 at the time this 
research was initiated). Nonetheless, this focus 
group was composed of the participants who met 
the required profile out of 64 participants who 
answered the prior information questionnaire. 

When people volunteer to participate in a 
study like this, they do not understand the im¬ 
portance of their commitment to the successful 
carrying out of a project. This leads to the fact that 
complying with their responsibility was not the 
participants’ priority, as it was for the researchers 
who depended on participants’ willingness and 
time availability to answer and pilot the tests, 
and had to accommodate our schedule to theirs. 
Moreover, some of the interviewees’ personal 
conditions made us fall behind schedule in 
scoring the tests, in classifying and analyzing the 
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data. On more than one occasion, the researchers 
had to deal with interviewees’ personal difficulties 
and problems. Another issue was that the only 
time that interviewees had was from 4 to 8 p.m. 
since most of them worked during the day and 
only arrived after 4:00 p.m. at the University. 

Due to the lack of model literature tests that 
covered most of the aspects students wanted to be 
evaluated, we had to spend more time in designing 
them. Due to the nature of the tests developed for 
this project, they had to be teacher corrected and 
could not be machine scored. 

After concluding the research we have the 
following questions for further study: How can 
the information obtained from the participants’ 
interviews be used to modify or improve the 
University’s current literature programs? How 
can the two proficiency tests resulting from this 
research be actually used by the University? Should 
they be revised or modified; if so, by whom? 
Should a question bank be developed in order 
not to repeat the exact test every semester? How 
can students’ abilities and competences be taken 
into consideration to improve the University’s 
literature programs? Can there be another way of 
accrediting previous knowledge? 

The American and British literature tests 
were designed based on the students’ needs, the 
University requirements and program, theory 
about literature and testing, and commercial tests 
to accredit previous knowledge. Competences and 
good command in English were essential factors 
in passing the tests. The tests were not published 
in this research to enhance its validity as they 
are going to be used by the University to accredit 
students’ previous knowledge. 
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Appendix 1: Prior Information Questionnaire 


The objective of this questionnaire is to establish which students, if any, of those studying Bilingual Edu¬ 
cation at the University, have any prior knowledge in the subjects of American and British Literature. If 
they do have this prior knowledge, we want to find out where and when they obtained it; and to ask if they 
want this knowledge to be accredited through a proficiency test. 

If you are interested, it would be very valuable for our research project, if you would answer the following 
questions: 


Personal Data: 

Name:_ 

Date:_Semester:. 


1. Do you have any previous knowledge on American Literature? Yes_No_ 

2. How would you consider your level in this area? 

High = able to identify authors, writings, literary periods, and historical context and also to do 
reading comprehension activities about them. High?_ 

Medium = able to identify most authors, writings, literary periods and do some reading compre¬ 
hension activities about them. Medium?_ 


Low = only identify a few authors and their work, but are not sure about literary periods or his¬ 
torical context. Not totally sure about doing reading comprehension activities. 

Low?_ 

3. Where did you obtain this knowledge? School?_ University?_ Other?_ 

Which?_ 

4. When did you obtain this knowledge? Approximate Year?_ 

5. Do you have any previous knowledge on British Literature? Yes_No_ 
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6. How would you consider your level in this area? *Use the same criteria as in question 2 

High? _ 

Medium? _ 
Low?_ 


7. Where did you obtain this knowledge? School? 

School? University? Other? 

Which? 


8. When did you obtain this knowledge? 

Approximate Year? 


9. Would you consider taking a proficiency exam to validate this knowledge? Yes_No 
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Appendix 2: Interview Format Used to Find Out the Participants' 
Opinions and Beliefs about Proficiency Tests 


Taking into account the answers you gave in the prior information questionnaire regarding literature 
proficiency tests that you answered on April 10 and 23 , 2007 , we would be very grateful if you would reply 
the following questions: 


1. Would you like to take the proficiency test for American Literature? Why? 

2. Would you like to take the proficiency test for British Literature? Why? 

3. Would you like to take it/them during the first semester? Why? 

4. If you have already started the program of Bilingual Education, when would you like to take it/ 
them? 

5. What kind of test would you like to take: 

- Short Open-ended questions 

- Closed questions e.g. fill in the blanks, multiple choice or true and false. 

- Reading comprehension 

- Essay type 

- Other?_Which one_ 


6. Do you think that the test should be focused on? 

- Authors 

- Topics 

- Historical Periods 

- Different genres of texts 

- A combination of the above 

- Other_Which one_ 


7. How many questions do you consider that this test should have? From: 

- 1-10 
- 1-20 

- 1-50 

- 1-100 

- Other?_Why?_ 


8. What do you remember about your previous literature studies? 
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Appendix 3: Questionnaire Used after 
the Proficiency Tests Were Administered 


Name: 

Test: 


Part A 

Independently of the results you achieved on the proficiency literature test you took, we would appreciate 
if you would answer the following questions truthfully and objectively. 


1. Did you study for the test? 

a. None 

b. A little 

c. More or less 

d. Consciously 


2. How do you think you did on the test? 

a. Excellent 

b. Acceptable 

c. Poorly 

Why? 


3. What was your mental and physical disposition at the moment of taking the test? 

a. Excellent, healthy, rested, optimistic 

b. Tired, not well, sick, negative 

c. Other? Explain 


4. How did you feel when you answered the literature proficiency test? 

a. Relaxed 

b. Comfortable 

c. Nervous 


PROFILE Vol. 12, No. 1, 2010. ISSN 1657-0790. Bogota, Colombia. Pages 159-179 


177 










Leon Chica, D'Costa Martinez & Franco Jacome 


5. Did you think the test was? 

a. Easy 

b. Normal 

c. Difficult 

6. Regarding the allotted time (3 hours) was it? 

a. Not enough 

b. Sufficient 

c. Adequate 

d. Excessive 

7. About the topics and questions, did the test: 

a. Meet your expectations? 

b. Was it too general? 

c. Was it too specific? Explain your answer. 


8. Grade the type of question from 1 to 5 being: 

1 = very easy, 2 = easy, 3 = average, 4 = difficult, and 5 = very difficult. 


Time Line 
10 points 

Multiple 

Choice 

30 Points 

Short Open-ended 
Questions 

10 Points 

Reading 
Comprehension 
20 Points 

Essay 

30 

Points 







9. Observations: in your own words give us any feedback or suggestions to improve the test. 
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Part B 

After knowing the results you obtained in the proficiency test, please answer the following two questions: 

10. Grade the type of question from 1 to 5 being: 

1 = very easy, 2 = easy, 3 = average, 4 = difficult and 5 = very difficult: 


Time Line 
10 points 

Multiple 

Choice 

30 Points 

Short Open-ended 
Questions 

10 Points 

Reading 

Comprehension 

20 Points 

Essay 

30 Points 







11. Observations: 

In your opinion, what part of the test would you modify? 
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